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PREFACE 


This  report  was  created  for  the  F-16  Aircrew  Training  De¬ 
velopment  Project  contract  no.  F02604-79-C8875  for  the  Tactical 
Air  Command  to  comply  with  the  requirements  of  CDRL  nos.  8029, 
8032,  and  B033.  The  project  entailed  the  design  and  development 
of  an  instructional  system  for  the  F-16  RTU  and  instructor 
pilots.  During  the  course  of  the  project,  a  series  of  develop¬ 
ment  reports  was  issued  describing  processes  and  products.  A 
list  of  those  reports  follows  this  page.  The  user  is  referred  to 
Report  No.  34,  A  Users  Guide  to  the  F-16  Training  Development 
Reports,  for  an  overview  and  explanation  of  the  series,  and 
Report  No.  35,  F-16  Final  Report,  for  an  overview  of  the  Instruc¬ 
tional  System  Development  Project. 
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EXECUTIVE  SUMMARY 


l 


The  computer  managed  instructional  (CMI)  system  required  for 
the  F-16  training  program  must  include  components  for  automatic 
testing,  record  keeping,  student  scheduling,  inventory  tracking, 
and  instructional  prescriptions.  The  testing  components  must 
provide  facilities  for  test  grading,  diagnostic  testing,  and  test 
security,  as  well  as  adequate  student  feedback  capability.  It 
must  also  permit  the  evaluation  of  instructional  effectiveness 
and  facilitate  course  revision.  The  student  records  component 
must  encompass  bibliographic  information,  training  performance 
data,  report  generation  of  records,  and  multiple  sorting  capabil¬ 
ities.  The  scheduling  component  must  provide  the  capabilities 
for  the  optimized  scheduling  of  student  learning  activities  and 
training  resources.  This  scheduling  capability  must  exist  at  the 
levels  of  syllabus  and  weekly  and  daily  timetables.  The  inven¬ 
tory  component  must  provide  short-  and  long-term  information  on 
all  resource  utilization  and  availability,  including  personnel, 
equipment,  facilities,  and  supplies.  The  prescriptive  component 
must  provide  the  capability  to  generate  individualized  student 
learning  schedules  based  upon  student  performance  data,  as  well 
as  training  resource  availability  and  utilization.^ 


Five  existing  instructional  systems  which  could'' serve  as  the 
basis  for  the  F-16  CMI  system  are  surveyed.  For  each  of  the 
systems  a  functional  description  and  examples  of  input/output  are 
provided.  The  five  systems  studied  were  AFORMS,  TICCIT,  VTS , 

Navy  CMI,  and  AIS.  None  of  the  five  systems  surveyed  could 
satisfy  all  of  the  capabilities  required  in  the  functional  speci¬ 
fications  for  the  F-16  training  environment,  and  the  limitations 
of  each  are  mentioned. 


A  cost  benefit  analysis  considered  the  general  cost  benefits 
applicable  to  the  F-16  environment.  These  include  savings  in  the 
amount  of  training  time  required,  more  effective  utilization  of 
training  resources,  improved  quality  control,  increased  standard¬ 
ization  and  flexibility  in  the  training  system,  and  greater  field 
deployability  and  expandibility .  Specific  cost  considerations 
for  the  F-16  program  were  made  by  extrapolating  from  data  on  the 
F-4/F-15  training  programs.  The  specific  cost  savings  projected 
on  this  basis  were  a  time  reduction  of  two  weeks  in  course  dura¬ 
tions,  or  25  student  salaries  for  the  training  period;  10  man 
years  of  instructor  time  per  course;  and  17  full-time  administra¬ 
tive  support  personnel. 


v 


The  conclusion  of  the  study  considered  the  five  systems 
surveyed  in  the  report  in  terms  of  functional  capabilities, 
operational  status,  remote  capability,  training  status,  and  cost 
factors.  Although  any  of  the  five  systems  could  be  used  as  a 
basis  for  the  F-16  CMI  system,  it  was  recommended  that  VTS  is 
presently  the  most  suitable  system,  because  it  a)  meets  nearly 
all  of  the  functional  requirements,  b)  is  relatively  inexpensive 
c)  is  basically  a  "stand-alone"  system,  d)  is  currently  in  use 
for  aircrew  training  and  is  an  approved  military  training  device 
and  e)  has  sufficient  scope  to  support  a  full-scale  training 
system. 
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COMPUTER  MANAGED  INSTRUCTION 
FOR  THE 

F-16  TRAINING  PROGRAM 


1 . 0  INTRODUCTION 


This  study,  representing  the  deliverable  for  CDRL  B032, 
consists  of  four  major  parts:  (1)  a  functional  specification  of 
the  F-16  computer  managed  instruction  (CMI)  system  which  meets 
the  needs  outlined  in  the  F-16  Training  Management  System  Needs 
and  Design  Concept  Analysis;  (F-16  Aircrew  Training  Development 
Project  Report  No.  12,  September  1980,  prepared  by  O'Neal  and 
Smith);  (2)  an  analysis  of  existing  systems  which  could  serve  as 
the  basis  of  the  F-16  CMI  system;  (3)  a  cost  benefit  analysis 
which  serves  as  a  support  rationale  for  the  specification;  and 
(4)  conclusions  and  recommendations  for  the  F-16  training 
program. 


2.0  F-16  CM I  FUNCTIONAL  SPECIFICATIONS 


This  CMI  system  shall  consist  of  five  management  components 
which  are  all  interrelated  and  interdependent.  These  components 
include  student  testing,  course  record  keeping,  scheduling, 
inventory  tracking,  and  prescription  of  new  instruction.  This 
section  provides  the  functional  specifications  for  each  of  these 
five  components. 


2 . 1  Testing  Component 

The  testing  component  performs  three  functions:  (1)  asses¬ 
ses  student  performance  level,  (2)  provides  student  feedback,  and 
(3)  provides  evidence  for  the  instructional  effectiveness  of  the 
course . 

2.1.1  Assessing  the  performance  level  of  the  student 
involves  a  test  grading  function,  a  diagnostic  subtesting 
capability,  and  control  features  which  minimize  the  possibility 
of  test  cheating. 

2. 1.1.1  Grading  Tests.  Automatic  test  grading  shall 
include  the  capability  to  read  and  grade  hard  copy  test  forms  and 
record/ store  these  grades.  It  shall  also  record  the  time 
required  to  accomplish  the  training  segment  and  the  number  of 
trials  to  criteria.  (Time  required  to  accomplish,  and  trials  to 
criteria  shall  establish  a  data  base  for  the  scheduling  component 
in  terms  of  estimating  student  rate  of  progress). 

2. 1.1. 2  Diagnostic  Testing.  A  diagnostic  testing  function 
shall  be  included.  This  function  will  allow  sub-testing  to 
explore  student  weakness  areas  in  terms  of  specific  skills  not 
mastered.  This  allows  effective  treatment  of  student  misunder¬ 
standings  and  reduces  the  amount  of  remedial/review  instruction 
required.  (This  information  is  input  to  the  prescriptive 
component . ) 


2. 1.1. 3  Control  Feature.  In  order  to  ensure  the  integrity 
of  student  performance  information,  control  features  shall  be 
included  in  the  system.  These  features  are  essential  in  an 
individualized  instruction  environment  since  tests  are  likely  to 
be  taken  at  different  times.  Hence,  cheating  can  be  a  problem. 
These  control  features  shall  involve  the  accumulation  of  a  large 
pool  of  test  items  for  each  objective  and  the  random  selection  of 
items  from  this  pool  to  produce  test  forms.  This  mechanism 
(i.e.,  random  selection  from  a  large  pool)  produces  an  individu¬ 
alized  test  for  each  student  (or  possibly  for  each  class  of 
students)  and  minimizes  the  problem  of  cheating  in  an  individual¬ 
ized  instruction  environment. 

2.1.2  Providing  feedback  to  students  is  an  essential 
feature  of  a  CMI  environment.  Because  students  are  working  on 
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their  own  individualized  programs  at  their  own  rates,  feedback  is 
more  critical  than  in  group-paced  instruction.  This  feedback 
shall  indicate  the  test  items  and  review/remedial  instruction 
required  (diagnostic  testing  capability).  The  feedback  shall 
occur  in  real-time  (i.e.,  the  feedback  should  be  produced  as  the 
tests  are  graded  and  recorded).  The  feedback  shall  also  indicate 
the  number  of  attempts  to  achieve  criterion,  the  elasped  time, 
and  the  projected  time. 

2.1.3  The  system  shall  provide  evidence  of  the  effective¬ 
ness  of  course  segments  in  terms  of  group  and  individual  per¬ 
formance  data.  This  capability  will  allow  the  general  improve¬ 
ment  of  the  instructional  effectiveness  of  the  course  by  identi¬ 
fying  inadequate  instructiona 1  units  or  media.  This  feature 
shall  involve  the  production  of  reports  which  show  cumulative 
student  responses  according  to  course  segments  in  terms  of  number 
of  students  passing/ failing  after  successive  attempts.  Test 
items  which  have  high  miss  rates  will  be  flagged.  The  feature 
shall  also  produce  reports  which  indicate  whether  review/remedial 
instruction  prescribed  was  effective  in  identifying  and  eliminat¬ 
ing  problems  for  individual  students. 


2.2  Student  Records  Component 

Information  contained  in  the  student  records  will  be  used  to 
evaluate  the  instructional  progress  of  the  student,  provide  feed¬ 
back  on  progress,  indicate  effectiveness  of  instructional  compon¬ 
ents,  and  generate  individualized  learning  prescriptions  for  each 
student.  To  meet  these  needs  the  system  shall  includes 

a.  Biographical  information  on  each  student  which  includes 
entry-level  information,  personnel  data,  and  previous 
job/ training  experience. 

b.  Training  performance  information  on  objectives  mastered 
including  results  on  each  test  item  administered,  number 
of  attempts /tr ia Is  for  each  objective,  and  time  required 
to  successfully  complete  objectives. 

c.  Report  generation  of  desired  information  stored  in 
records  as  required  at  different  levels  of  course 
management . 

d.  Multiple  sort  capability  with  real-time  response. 


2.3  Scheduling  Component 

The  capability  to  schedule  learning  activities  and  resources 
in  the  most  optimal  manner  possible  is  fundamental  to  any  effec¬ 
tive  system.  For  the  F-16  training  system,  the  schedule  must 
operate  at  three  levels:  syllabus,  weekly,  and  daily. 
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2.3.1  At  the  syllabus  level,  it  shall  be  designed  to  assess 
the  adequacy  of  the  facility  for  the  projected  student  through¬ 
put.  Using  the  inventory  of  resources  and  the  student  record 
data,  it  should  indicate  (a)  the  training  resources  available 
versus  those  required,  (b)  training  accomplishments  versus 
projected  graduation  dates,  and  (c)  provide  an  analysis  of 
individual  student  progress  in  terms  of  dates  for  segments  to  be 
completed . 

2.3.2  At  the  weekly  level,  the  scheduling  function  shall  be 
designed  to  assess  daily  training  resources  and  personnel  avail¬ 
ability  in  terms  of  (a)  short-term  projected  schedules  which  name 
specific  training  events,  gunnery  ranges,  or  aircraft  and  (b) 
ability  to  evaluate  student  progress  in  terms  of  predicting  where 
in  the  course  the  student  will  be  and  whether  needed  resources 
will  be  available  at  that  time. 

2.3.3  At  the  daily  level,  it  shall  be  designed  to  incorpor¬ 
ate  the  ability  to  "fine  tune"  the  training  schedule  around  daily 
occurrences  which  generate  exceptions  to  the  weekly  schedule. 

This  will  involve: 

a.  Ensuring  that  pilots  (IPs  and  students)  comply  with 
command  regulations  and  local  constraints  regarding 
prerequisite  events,  crew  rest,  medical  clearances, 
and  currency  prerequisites,  and  general  supervisory 
considerations . 

b.  Adjusting  daily  schedules  to  allow  for  missed  or  unsat¬ 
isfactory  events  (i.e.,  aborts,  weather,  etc.),  and 
providing  necessary  notification  for  rescheduling  of 
affected  personnel. 


2.4  Inventory  Component 

The  F-16  CM I  system  shall  maintain,  on  a  real-time  basis, 
current  inventories  and  usage  records  on  all  training  resources 
in  order  to  detect  deficiencies  and  nonefficient  usage.  These 
inventories  shall  be  accessible  by  the  report  generation  capa¬ 
bility  of  the  student  records  component.  Inventories  shall  be 
kept  on  training  personnel,  facilities,  equipment,  and  supplies. 

2.4.1  Personnel  shall  be  inventoried  as  a  training  system 
resource  on  a  long-term  and  day-to-day  availability  basis: 

a.  Long-term  tracking  shall  concern  the  following  informa¬ 
tion  as  a  minimum:  Aircraft  qualification,  instructor 
speciality,  expected  date  of  full  qualification,  and 
proposed  rotation  date. 

b.  Short-term  availability  will  be  concerned  with  informa¬ 
tion  on  projected  leave,  days  of r.  and  scheduled  activi¬ 
ties  of  all  personnel. 


2.4.2  All  training  facilities  such  as  classrooms,  briefing/ 
conference  rooms,  and  study  carrels  shall  be  inventoried.  Facil¬ 
ities  shall  be  identified  by  student  capacity,  audiovisual  capa¬ 
bility,  time  availability,  and  current  status. 

a.  Student  capacity  will  be  based  upon  the  type  of  training 
to  be  accomplished  and  the  media  to  be  employed. 

b.  Audiovisual  capability  shall  include  study  room,  sound/ 
slide  (individual  and  group),  videotape  (individual  and 
group),  or  computer  assisted  instruction  (CAI). 

c.  Time  availability  shall  indicate  the  regularly  scheduled 
use  of  the  facility  and  the  duration  of  that  use. 

d.  Current  status  should  indicate  whether  the  facility  is 
in  use,  in  maintenance,  or  open. 

2.4.3  All  training  equipment  (e.g.,  simulators)  shall  be 
inventoried.  This  information  shall  include  the  hard  and  flex¬ 
ible  schedules  for  use,  daily  demands  on  the  equipment,  scheduled 
maintenance,  and  current  status. 

2.4.4  All  supplies  required  in  the  training  program  shall 
be  inventoried.  This  includes  administrative  materials  (e.g., 
paper,  pencils,  forms),  as  well  as  operational  supplies  (e.g., 
fuel,  ordnance).  The  inventory  must  indicate  the  existing  and 
planned  usage  and  generate  reports  indicating  over-  or  underuse 
limits.  The  status  of  reorders  must  also  be  available  to  check 
on  late  orders. 


2.5  Prescriptive  Components 

Given  the  learning  characteristics  of  the  student  and  previ¬ 
ously  mastered  skills/tasks,  a  new  instructional  task  or  activity 
must  be  prescribed  after  each  test  or  performance  evaluation 
point.  This  capability  can  range  from  a  "canned"  prescription  in 
which  each  event  has  a  fixed  activity,  to  a  fully  dynamic  situa¬ 
tion  where  course  performance  and  resource  availability  are  con¬ 
sidered  in  making  the  prescription.  The  F-16  CMI  system  shall: 

a.  Generate  individual  learning  prescriptions  for  each 
student  which  includes  objectives  or  tasks  to  be  com¬ 
pleted,  mastery  levels  for  satisfactory  completion  of 
these  objectives/tasks,  and  a  timetable  for  learning 
activities . 

b.  Make  prescriptions  which  optimize  the  utilization  of 
available  resources  and  which  account  for  scheduling 
changes,  or  personnel  availability. 

c.  Allow  the  students  to  interrogate  the  system  for  further 
information  in  terms  of  additional  resources  or  addi¬ 
tional  instructional  activities. 


d.  Allow  students  choice  of  activities,  media,  or  sequenc¬ 
ing,  where  feasible,  and  record  this  choice  of  alterna¬ 
tives  . 


2.6  Summary  of  F-16  CMI  Specifications 

The  F-16  CMI  system  shall  consist  of  five  major  components, 
defined  in  terms  of  a  number  of  essential  features  for  each  com¬ 
ponent.  The  testing  component  shall  include  functions  to  grade 
tests,  perform  diagnostic  testing,  and  control  test  security.  It 
shall  also  provide  the  capability  for  student  feedback,  as  well 
as  monitor  the  instructional  effectiveness  of  the  course.  The 
student  record  component  shall  include  biographic  and  student 
performance  data  base  functions,  as  well  as  report  generation  and 
file  sorting  capabilities.  The  scheduling  component  must  be  able 
to  schedule  at  the  syllabus,  daily  and  weekly  levels.  The  inven¬ 
tory  component  shall  maintain  records  on  the  availability  of  all 
training  personnel,  facilities,  equipment,  and  supplies  and  also 
allow  projections  of  resource  utilization.  Finally,  the  pre¬ 
scriptive  component  shall  generate  individual  learning  schedules, 
optimize  the  utilization  of  resources,  and  allow  students  to 
interrogate  the  system  for  further  information,  or  to  exercise 
choice  of  alternatives,  when  possible.  These  five  components 
provide  the  minimal  CMI  support  capabilities  necessary  for  the 
effective  management  of  the  F-16  training  program  and,  also,  to 
satisfy  the  needs  analysis  completed  earlier. 


3.0  ANALYSIS  OF  EXISTING  SYSTEMS  SUITABLE  FOR  CMI 


A  number  of  existing  systems  could  serve  as  the  basis  for 
the  development  of  a  CMI  system  for  the  F-16  training  program. 
This  section  provides  a  brief  functional  description  of,  and 
examples  of  program  descriptions  and  input/output  (where  avail¬ 
able)  for  five  such  systems:  Air  Force  Operations  Resource 
Management  System  (AFORMS) ,  Time-shared  Interactive  Computer 
Controlled  Information  Television  (TICCIT)  ,  Versatile  Training 
System  (VTS) ,  Navy  CMI,  and  Advanced  Instructional  System  (AIS) . 
While  these  are  not  the  only  existing  systems  with  the  capabili¬ 
ties  required  to  meet  the  functional  specifications  for  the  F-16 
CMI  system,  they  are  considered  the  best  candidates  in  terms  of 
cost/benefit  considerations.  For  those  systems  where  sample 
output  or  program  descriptions  are  available,  the  discussion  is 
abbreviated . 


3.1  Analysis  of  Capabilities  of  AFORMS  for  CMI 

The  AFORMS  was  designed  to  fill  the  need  for  better  tracking 
and  monitoring  of  aircrew  training  programs.  The  overall  objec¬ 
tive  of  AFORMS  is  to  identify,  compile,  track,  and  report  func¬ 
tional  management  information  needed  to: 

1.  Support  implementation  of  USAF  regulation  flight 
management  policies, 

2.  Provide  accurate,  timely  aircrew  data  to  assist  in  the 
determination  and  reporting  of  mission  capabilities, 
combat  readiness,  and  aircrew  experience,  and 

3.  Ensure  ready  availability  of  aircrew  information  is 
readily  available  in  formats  that  allow  decisions  and 
accurate  tracking  of  training  accomplishments. 

It  is  intended  that  AFORMS  provide  an  online,  interactive 
and  real-time,  AF-wide  management  system  with  common  data  bases, 
reports,  and  centralized  system  development/maintenance  work 
while  still  accommodating  the  different  needs,  organizational 
structures,  and  existing  DP  systems  of  the  various  AF  commands. 

In  addition,  it  is  proposed  that  current  (batch  oriented)  hard¬ 
ware  be  used  as  the  basis  for  the  system.  Figure  1  outlines  the 
nature  of  the  input/output  relationships  in  AFORMS,  as  given  in 
the  AFORMS  functional  description  document,  December  1976. 

The  initial  implementation  will  provide  two  major  systems  to 
meet  three  general  objectives  of  AFORMS.  These  are  the  resource 
management  and  aircrew  training  management  systems.  Resource 
management  involves  the  maintenance  of  an  accurate  inventory  of 
assigned  resources,  project  resource  availability  and  utiliza¬ 
tion,  and  the  documentation  of  actual  resource  usage.  The 
primary  resources  involved  are  aircrew  personnel  and  aircraft 


Figure  1.  Complete  A  FORMS  System 
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flying  hours.  Table  1  summarizes  the  specific  performance 
requirements  necessary  for  resource  management  support.  Aircrew 
training  management  involves  the  maintenance  of  an  accurate 
inventory  of  individuals  and  crews  who  require  aircrew  training, 
a  determination  of  the  specific  training  required  for  each 
individual  or  crew,  and  the  maintenance  of  records  of  the  actual 
training  accomplished  by  each  individual  or  crew.  Table  2 
summarizes  the  specific  performance  requirements  required  for 
aircrew  training  management  support.  For  each  of  the  components 
given  in  Tables  1  and  2,  there  is  a  corresponding  data  base 
involving  between  10  and  100  separate  data  items.  There  are  23 
types  of  standardized  reports  generated. 

The  current  form  of  AFORMS  could  be  used  as  a  basis  for  a 
CMI  system;  however,  the  present  implementation  does  not  contain 
the  fundamental  capabilities  needed  for  CMI,  i.e.,  prescription, 
scheduling,  and  evaluation  functions.  Thus  it  would  be  necessary 
to  provide  functions  to  generate  individualized  aircrew  training 
prescriptions  and  timetables,  as  well  as  automatic  test  grading 
and  diagnostic  testing.  AFORMS  does  not  have  provisions  for 
actual  aircrew  scheduling  (i.e.,  individual  or  crews)  or  facili¬ 
ties  for  compiling/manipulating  historical  data  bases  for  student 
records  or  resource  utilization.  It  does  have  the  components 
necessary  to  generate  such  schedules  or  keep  student  records 
(i.e.,  aircraft  assignment,  continuation  training  assignment,  and 
continuation  training  event  recording).  Building  a  CMI  system 
using  AFORMS  requires  extending  these  capabilities.  AFORMS  does 
have  a  report  generation  mechanism  which  would  be  used  in  the 
generation  of  learning  prescriptions  and  evaluation  activity. 

There  are  a  number  of  implementation  problems  with  the  use 
of  AFORMS  for  CMI:  The  date  the  system  will  be  fully  operational 
is  unknown;  CMI  capabilities  will  probably  have  low  development 
priority  in  AFORMS;  and,  most  importantly,  AFORMS  is  planned  to 
be  used  on  the  already  saturated  base  Burroughs  3500  computer 
which  will  greatly  inhibit  the  required  real-time  response. 


3.2  Analysis  of  TICCIT  CMI  Capabilities 

TICCIT  is  a  fully  operational,  interactive  training  system 
owned  by  Hazeltine  Corporation  and  developed  jointly  by  MITRE  and 
Brigham  Young  University.  TICCIT  is  a  stand-alone  minicomputer- 
based  system  supporting  up  to  128  terminals  per  system.  TICCIT 
technology  is  based  upon  color  television  displays  of  intermixed 
alpha/numerics,  stored  graphics,  motion  video,  and  random  access 
audio,  with  user  response  via  keyboard  and  light  pen.  TICCIT 
facilities  include  a  complete  range  of  authoring  features,  data 
entry  aids,  data  base  management  capabilities,  and  record  acqui¬ 
sition/analysis  utilities.  Although  TICCIT  is  primarily  designed 
for  CAI  activities,  the  facilities  are  suitable  for  CMI  support. 
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Table  1 


Performance  Requirements  for  Resource 
Management  Support  in  AFORMS 


Component 

Personnel  Inventory  and 
Status 

Aviation  Career 
Incentive  Act 


Aircrew  Flying 
Exper ience 


Capabilities 

Store  and  maintain  identification 
and  status  of  all  AF  personnel. 

Store  and  maintain  data  necessary 
to  record  accumulated  months  of 
flying  duty. 

Store  and  maintain  qualification 
data  and  total  flying/sortie  hours 
for  all  aircraft  types. 


Crew  Inventory  and  Store  and  maintain  identification 

Status  and  status  for  each  integral  crew. 


Aeronautical  Orders 


Generate  and  display/print 
Aeronautical  Orders. 


Data  Transfer  for 
Change  of  Station 


Management  Support 
During  Deployment 


Flying  Hour  Allocations 


Military  Personnel 
Authorizations 

Reserve  Personnel 
Authorizations 


Transfer  all  identification,  and 
status  data  for  any  personnel 
transferred . 

Transfer  all  identification  and 
status  data  for  personnel  deployed 
for  flight  duty. 

Store  and  maintain  data  of 
allocation  of  all  aircraft  types. 

Store  and  maintain  authorization 
and  tour  type  data. 

Store  and  maintain  authorization 
and  tour  type  data. 


Inventory  and  Status  of 
Personnel 


Store  and  maintain  identification 
and  status  of  all  personnel  on 
active  flying  status. 


Table  2 


m 


Performance  Requirements  for 
Aircrew  Training  Management  Support  in  AFORMS 


Component 


Capabilities 


Training  Requirements 
Management 


Aircraft  Assignment  for 
Continuation 


Continuation  Training 
Assignment  (Individual) 


Continuation  Training 
Event  Recording 
(Individual) 


Store  and  maintain  training  event 
description  and  requirements  tables 
for  all  types  of  aircraft,  crew 
positions,  mission  types  and 
proficiency  levels. 

Store  and  maintain  identification 
and  status  data  for  combination  of 
all  aircraft  and  crew  position 
types  required. 

Store  and  maintain  identification 
and  status  for  each  continuation 
training  plan  an  individual  is 
required  to  complete. 

Store  and  maintain  identification, 
requirement,  and  summary 
accomplishment  data  required  for 
all  individuals. 


Aircraft  Assignment  for 
Continuation  Training 
(Crew) 

Continuation  Training 
Assignment  (Crew) 


Continuation  Training 
Event  Recording  (Crew) 


Aircraft  Assignment  for 
Qualification  Training 
(Individual) 


Qualification  Training 
Assignment  (Individual) 


Store  and  maintain  identification 
and  status  for  each  type  aircarft 
in  terms  of  crew  requirements. 

Store  and  maintain  identification 
and  status  data  for  each  type 
aircraft  in  terms  of  crew 
requirements . 

Store  and  maintain  identification, 
requirement,  and  summary 
accomplishment  data  required  for 
all  crews. 

Store  and  maintain  identification 
and  status  for  combinations  of 
aircraft  and  crew  positions 
required  for  qualification  training 
plans . 

Store  and  maintain  identification 
and  status  of  individual 
assignments  for  qualification 
training  plans. 
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Table  2  (continued) 


Component 


Capabilities 


Qualification  Training 
Event  Recording 
(Individual) 


Ground  Training 
Assignment 


Ground  Training  Event 
Record ing 


Store  and  maintain  identification, 
requirement  and  summary 
accomplishment  data  for  each  event 
individuals  must  perform  in 
qualification  training. 

Store  and  maintain  identification 
and  status  data  for  each  ground 
training  plan. 

Store  and  maintain  identification, 
requirement,  and  summary 
accomplishment  data  for  each  event 
individuals  must  perform  in  ground 
training . 


The  TICCIT  software  system  includes  a  registration  package 
which  permits  the  creation  of  an  identification  number  and  pass¬ 
word,  assignment  of  an  authorization  mode/level  (e.g.,  student, 
instructor),  sorting  of  the  student's  name  into  an  online  user 
directory,  and  the  modification  or  deletion  of  all  registration 
information.  TICCIT  software  includes  capabilities  to  construct, 
alter,  and  deliver  tests,  either  online  or  offline,  as  well  as 
automatic  test  analysis  and  grading  features.  Controls  on  test 
security,  such  as  random  selection  of  test  items  from  an  item 
pool  are  available  in  the  software.  Diagnostic  testing  of  a 
limited  nature  is  also  possible.  In  general,  TICCIT  allows 
considerably  more  sophisticated  testing  than  the  usual  multiple 
choice/mark  sense  grading  characteristic  of  other  systems. 

All  student  interactions  may  be  recorded  automatically, 
allowing  the  tracking  of  any  individual  student  (on  a  segment-by¬ 
segment  or  response-by-response  basis)  or  any  group  of  students. 

A  file  sorting  capability  allows  the  manipulation  of  student 
data  according  to  any  or  all  fields  in  the  records.  Report 
generation  capabilities  allow  the  automatic  production  of  reports 
on  test  statistics  (e.g.,  item  analyses),  terminal  usage,  and 
student  performance  statistics.  The  graphics  capability  of 
TICCIT  also  permits  the  online  monitoring  of  student  progress  or 
resource  usage  data  using  color-coded  charts,  graphics  or  block 
diagrams.  TICCIT  software  has  a  built  in  "advisor"  feature  which 
would  be  suitable  for  the  prescriptive  capabilities  required  in 
CMI  with  minor  adaptions.  The  software  also  has  built-in 
features  for  providing  students  with  feedback  and  helps  which 
would  be  useful  in  the  prescriptive  function  of  CMI. 

From  a  CMI  point  of  view,  the  TICCIT  system  needs  enhance¬ 
ment  in  the  capabilities  for  scheduling  and  inventory.  The 
features  needed  for  producing  individual  timetables  or  equipment 
schedules,  or  for  projecting  resource  needs/usage,  are  limited  in 
the  standard  TICCIT  software  system,  although  the  data  base  man¬ 
agement  and  report  generation  components  which  do  exist  could  be 
modified  to  encompass  these  functions.  Examples  of  TICCIT 
program  descriptions  and  sample  output  are  given  in  Appendix  A. 


3.3  Analysis  of  VTS  CMI  Capabilities 

VTS  is  a  training  support  system  oriented  toward  aircraft 
maintenance  training  which  encompasses  a  range  of  CMI  and  Person¬ 
nel  Qualification  Statements  (PQS)  functions.  It  is  a  standard 
Navy  training  device  (cog.  20  #11869  and  11869A)  and  is  also  used 
in  an  Army  training  application  at  Fort  Sill,  Oklahoma. 

The  capabilities  for  VTS  when  fully  implemented  are  shown  in 
Figure  2.  These  include  the  following  functions: 

1.  Determine  and  select  the  best  possible  billet  for  each 
person  based  on  their  individual  capabilities,  prior 
experience,  and  training. 
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Figure  2.  Complete  Versatile  Training  System  (VTS) 


2.  Tailor  an  individual's  training  by  comprehensive  testing 
(i.e.,  diagnostic,  pretesting,  and  posttesting), 
individualized  prescription,  and  path  specification. 

3.  Provide  individualized  instruction  under  ISD  guidelines. 

4.  Enable  a  course  manager  to  author,  edit,  review,  and 
update  training  materials  online. 

5.  Provide  software  which  will  enable  an  instructor  to 
design,  update,  monitor,  and  control  trainers/simulators 
in  a  time-shared,  high-level  language. 

6.  Schedule  training  resources  (aircraft,  simulators, 
maintenance  trainers,  personnel,  etc.)  to  meet  total 
training  requirements  and  priorities. 

7.  Schedule  training  while  holding  consumption  of  resources 
(personnel,  training  equipment,  time,  and  material)  to  a 
minimum. 

8.  Provide  instructor  training  coordinators,  quota  control 
personnel,  and  supervisors  with  current  training 
progress  and  future  training  needs. 

9.  Provide  online  quota  control  capabilities  required  to 
support  officer  and  enlisted  personnel  training  for  all 
operational  squadrons. 

10.  Permit  consideration  of  the  effect  of  a  decision  in 
advance  by  supplying  complete,  accurate,  and  timely 
training  data  for  use  in  the  planning  and  decision¬ 
making  processes. 

11.  Eliminate  from  the  planning  and  decision-making  proces¬ 
ses  the  problems  associated  with  the  use  of  inconsistent 
and  incomplete  training  data  by  providing  a  means  of 
preparing  and  presenting  training  information  in  a 
uniform  manner. 

12.  Merge  resource  and  production  data  to  produce  signifi¬ 
cant  measures  of  training  performance,  facilitate 
control  of  costs,  and  facilitate  training  decisions  with 
minimum  processing  of  data. 

The  fully  implemented  VTS  system  will  consist  of  five  major 
software  components: 

3.3.1  Enlisted  Training  Software.  The  enlisted  training 
software  determines  enlisted  billets  which  are  vacant  within  the 
operational  squadrons,  or  soon  will  be,  and  matches  an  individual 
by  background  and  training  to  a  specific  billet.  After  billet 
selection  is  complete,  the  enlisted  training  software,  in  con¬ 
junction  with  the  resource  conf iguration  and  scheduling  software, 
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generates  a  training  program  to  qualify  the  person  for  the 
billet,  and  generates  the  necessary  forms  to  assign  the  person  to 
formal  training.  As  the  trainee  progresses  through  training,  the 
CMI  software  grades  him  or  her,  records  the  results,  and  provides 
the  necessary  prescription  to  enable  the  trainee  to  eliminate 
deficiencies  and  continue  training.  The  enlisted  training  soft¬ 
ware  then  produces  completion  letters  and  other  letters,  forms, 
and  reports  required  to  coordinate  training  between  the  various 
training  units.  The  enlisted  training  software  also  generates 
student  rosters,  rating  sheets,  and  other  documentation  records, 
such  as  special  training  reports  and  exception  reports. 

3.3.2  CMI /PQS/Testing  Software.  The  CMl/PQS/testing  soft¬ 
ware  will  accomplish  both  online  and  offline  diagnostic  and 
measurement  testing  functions  in  all  areas  of  VTS  activity.  This 
software  will  grade  tests,  maintain  statistical  data,  determine 
test  validity,  indicate  testing  trends,  and  produce  reports  rela¬ 
tive  to  these  functions.  Software  is  supplied  with  which  lesson- 
ware  can  be  added,  updated,  etc.,  according  to  users'  needs. 
Lessonware  will  be  introduced  into  the  VTS  in  a  hierarchical 
manner  to  accommodate  ISD.  Based  on  such  parameters  as  cost 
effectiveness,  equipment  and  instructor  availability,  billet 
criticality,  time  constraints,  and  the  extent  of  training 
required,  the  CMI  software  will  schedule  an  individual's  training 
within  ISD  guidelines.  The  training  will  be  broken  down  to  the 
learning  objective  level,  and  the  CMI  software  will  test  how  well 
the  student  has  learned  each  objective  by  means  of  online  tests. 
If  the  tests  indicate  a  need,  the  software  will  prescribe  remedi¬ 
ation  or  will  let  the  student  begin  work  on  the  next  learning 
objective.  The  CMI  software  will  also  generate  statistics  which 
grade  each  student,  the  instructor,  or  an  entire  class,  determine 
the  validity  of  test  questions,  and  check  other  relevant  para¬ 
meters  . 

3.3.3  Resource  Configuration  and  Scheduling  (RCAS)  Soft¬ 
ware.  The  RCAS  software  will  interact  with  other  VTS  software  to 
schedule  training  resources  in  accordance  with  training  needs. 

The  training  resources  would  include  aircraft,  simulators,  equip¬ 
ment,  classrooms,  instructors,  scheduled  classes,  etc.  RCAS  will 
monitor  the  status  of  resources  to  help  ensure  that  equipment  is 
available  and  operative  when  needed  and  is  properly  configured. 
This  aircraft  might  also  be  made  available  by  RCAS  software  to  an 
aircraft  maintenance  class.  Data  will  be  collected  by  RCAS  soft¬ 
ware  concerning  present  and  future  resource  requirements  in  order 
that  resources  can  be  effectively  managed. 

3.3.4  Reports  Geneiation  Software.  The  reports  generation 
software  will  provide  high-volume  production  of  forms,  letters, 
and  standardized  documents  automatically  utilizing  a  keyboard 
terminal,  video  monitor,  local  line  printer,  and  computer.  This 
software  will  allow  the  operator  to  input  and  display  a  full  page 
of  data  on  the  video  screen  in  a  format  that  is  operator  select¬ 
able.  Additions,  corrections  and/or  deletions  can  be  made  online 
and  the  page  proofread  and  fully  formatted  before  being  automat- 
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ically  typed  or  dumped  by  the  local  line  printer  or  highspeed 
printer. 

3.3.5  Aircrew  Training  Software.  The  aircrew  training 
software  prepares  an  individualized  training  program  for  each 
aircrew  member  after  comparing  his  training  history  with  a  stand¬ 
ard  training  syllabus.  This  syllabus  consists  of  a  list  of 
events  (tasks)  the  student  is  expected  to  perform  in  an  accurate 
and  timely  manner.  The  software  schedules  training  in  those 
tasks  in  which  the  student  is  deficient  and  predicts  a  training 
completion  date.  To  make  efficient  use  of  training  resources, 
the  aircrew  training  software  interacts  with  the  RCAS  software 
which  has  information  regarding  resource  availability  and  status. 
As  the  student  progresses  in  the  training  cycle,  student  and 
class  progress  reports  are  prepared  by  the  aircrew  training  soft¬ 
ware,  and  the  CMI  software  analyzes  the  work,  produces  grades, 
and  generates  prescriptions  to  allow  the  student  to  eliminate 
deficiencies  and  continue  on  to  the  next  event.  In  this  manner, 
the  aircrew  member's  training  is  directed  and  monitored  by  the 
software  until  completion,  at  which  time  completion  forms  are 
generated.  In  addition,  aircrew  training  software  performs  long¬ 
term  trend  analysis  to  spotlight  deficiencies  in  the  training 
program  itself,  in  order  that  the  program  can  be  updated  and 
optimized . 

Clearly  the  fully  implemented  VTS  system  will  provide  a 
comprehensive  set  of  CMI  capabilities,  far  more  than  are  required 
for  the  F-16  environment.  Thus  the  problem  with  VTS  is  not  one 
of  enhancing  or  adapting  a  system  for  CMI  functions,  but  rather 
of  selecting  the  specific  components  needed  for  the  F-16  system. 


3.4  Analysis  of  Navy  CMI  System  Capabilities 

The  Navy  CMI  system  was  developed  in  1968  by  the  Navy 
Personnel  Research  and  Development  Center  in  Memphis,  Tennessee. 
At  present,  it  provides  daily  instructional  management  at  four 
different  training  centers  for  more  than  6,000  students  in  nine 
different  courses.  It  is  expected  that,  when  fully  implemented, 
it  will  manage  about  13,000  to  15,000  students  in  24  courses. 

The  Navy  CMI  system  encompasses  capabilities  for  automated  test 
grading,  student  record  collection,  report  generation  and  pre¬ 
scription  generation. 

Testing  capabilities  involve  four  types  of  test:  progress 
tests,  remedial  tests,  unit  posttests,  and  performance  tests. 
Progress  tests  are  self-checked  tests  which  the  student  can  take 
as  soon  as  desired.  These  tests  are  not  used  as  a  basis  for 
grade  assignment  but  for  student  self  assessment.  Remedial  tests 
are  diagnostic  tools  which  are  tied  into  the  amount  of  remedial 
instruction  available  for  a  particular  unit.  They  prescribe 
specific  review/remedial  instruction,  tests,  or  activities.  Unit 
posttests  are  provided  as  evaluative  tools  in  terms  of  course 
improvement  and  student  retention  measurement.  Performance  tests 
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are  intended  to  be  used  to  assign  grades  and  evaluate  student 
progress . 

Optically  scored  answer  sheets  are  used  for  the  remedial 
tests,  posttests,  and  performance  tests.  After  the  answer  sheets 
are  graded,  the  system  tells  the  student  which  items  were  missed 
and  produces  feedback  in  the  form  of  a  remedial  study  plan  or 
test  assignment.  (Alternate  test  forms  are  used  to  minimize  the 
problem  of  cheating.)  The  performance  data  collected  from  the 
tests  is  accumulated  in  a  response  history  file  for  each  student 
and  used  to  generate  test  item  analysis  reports  and  a  daily 
student  progress  report.  The  system  also  produces  reports  which 
summarize  student  throughput  in  terms  of  number  of  graduates  or 
attritions,  their  total  time  on  the  course,  grade  levels 
achieved,  prospective  job  assignment  codes,  etc. 

With  respect  to  the  F-16  CMI  context,  the  Navy  CMI  would 
seem  to  provide  at  least  the  minimum  level  of  support  required  in 
the  functional  specifications.  The  system  does  appear  to  have 
adequate  test  grading,  student  record  collection,  and  diagnostic 
testing  capabilities.  The  report  generation  and  prescriptive 
capabilities  are  also  adequate,  although  there  is  a  need  for  add¬ 
itional  facilities  in  these  areas.  The  system  is  weak  in  terms 
of  inventory  and  resource  scheduling  capabilities.  Furthermore, 
the  system  technology  is  relatively  inefficiently  used  (i.e., 
batch  and  card  oriented),  and  experience  has  shown  that  it  is 
difficult  to  make  modifications  within  the  system  environment. 

Examples  of  input  forms,  individual  learning  prescriptions, 
and  output  reports  for  the  Navy  CMI  system  are  given  in  Appendix 
B . 


3.5  Analysis  of  AIS  for  CMI 

The  AIS  is  a  CMI  system  developed  at  Lowry  Air  Force  Base  by 
the  Human  Resources  Laboratory  and  McDonnell  Douglas.  While  it 
is  primarily  designed  as  an  operational  type  of  system,  it  is 
also  intended  to  provide  research  and  development  capability. 

AIS  is  still  in  the  prototype/experimental  phase  and  documenta¬ 
tion  is  scarce;  however,  because  of  its  potential  relevance,  a 
brief  description  is  included. 

AIS  provides  capabilities  for; 

1.  Individual  preassessment/diagnostic  testing. 

2.  Individual  progress  predictions  and  management. 

3.  Individual  instructional  assignments. 

4.  Online  test  scoring. 

5.  Automatic  resource  tracking  and  allocation. 
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6.  Automatic  data  storage  and  report  generation. 

AIS  is  a  real-time  system  based  upon  a  large  mainframe  (CDC 
Cyber)  with  capacity  to  support  up  to  50  CMI  terminals.  Each  CMI 
terminal  will  c  jist  of  plasma-type  cathode  ray  tube  (CRT) ,  an 
optical  mark-sti  ..ing  reader,  a  thermal  printer,  and  a  minicom¬ 
puter-based  controller.  AIS  will  use  the  CAMIL  language  for 
authoring  and  has  additional  file  handling  software  providing 
management  information  support. 

Because  documentation  on  AIS  is  sparse  it  is  difficult  to 
properly  assess  its  current  suitability  for  the  F-16  CMI  environ¬ 
ment.  It  appears  that  AIS  will  have  all  of  the  capabilities 
identified  in  the  functional  specification,  plus  additional 
research  capabilities  which  will  not  be  specifically  needed  in 
the  F-16  environment. 
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4.0  COST  BENEFIT  ANALYSIS 


4.1  General  Cost  Benefits  for  F-16  Environment 


A  CMI  system  c 
many  of  which  invol 
these  benefits  will 
particular  characte 
possibility  of  modi 
cost  savings  to  be 
resulting  from  CMI 
however,  they  will 
every  instructional 


an  result  in  a  number  of  training  benefits, 
ve  potential  cost  savings.  Whether  or  not 
produce  cost  savings  is  dependent  upon  the 
ristics  of  the  training  environment  and  the 
fying  the  present  environment  to  allow  the 
realized.  In  other  words,  the  benefits 
will  generally  apply  to  all  training  contexts; 
not  necessarily  result  in  cost  savings  for 
situation. 


Some  of  the  potential  benefits  of  CMI  applicable  to  the  F-16 
training  context  are: 

(a)  Time  savings.  A  major  benefit  is  the  possibility  of 
reducing  the  amount  of  training  time  required  through  individual¬ 
ized  instruction.  With  all  students  progressing  at  their  own 
rates,  the  faster  students  can  finish  sooner  and  the  slower  stu¬ 
dents  do  not  hold  up  the  others.  Since  training  time  typically 
means  lost  on-the-job  time,  this  time  savings  translates  into 
cost  savings.  The  Navy  CMI  system  reduced  course  lengths  by  46.8 
percent  resulting  in  an  annual  savings  for  FY  75  of  more  than  $10 
million  in  student  salaries.  AIS  projects  a  reduction  of  40 
percent  in  course  lengths  resulting  in  a  savings  of  645  student 
man-years.  This  saved  time  can  mean  higher  student  throughput. 


(b)  Effective 
instructional  reso 
equipment  (e.g.,  s 
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components.  These 
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esources  and  the  reduction  of  timetabling  mix- 
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.  For  example,  the  Navy  CMI  system  has  reduced 
rt  personnel  needed  by  23  percent  for  a  savings 
FY  75,  as  well  as  increasing  the  instructor  to 


(c)  Quality  control.  CMI  tends  to  improve  the  quality  con¬ 
trol  of  a  training  system  by  virtue  of  providing  closer  and  more 
exact  knowledge  of  student  progress.  It  is  possible  to  diagnose 
student  problems  sooner  and  also  to  detect  when  a  student  has 
successfully  mastered  a  skill  and  hence  is  •  ►  dy  to  progress  to 
another  task.  This  benefit  can  mean  cost  savings  in  those  cases 
where  inadequate  monitoring  of  student  progress  results  in  stu¬ 
dent  failure  (and,  therefore,  the  need  for  review  or  retraining). 
The  consequences  of  inadequate  quality  control  in  the  operation 
of  powerful  and  expensive  weapon  systems  are  rather  poignant 
(e.g.,  the  cost  of  a  destroyed  F-16  aircraft).  The  Navy  CMI  sys¬ 
tem  has  demonstrated  two  concrete  effects  of  increased  quality 
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control;  one  is  higher  performance  levels  on  exams,  the  other  is 
lower  student  attrition.  The  latter  factor  saves  money  in  terms 
of  reduced  waste  of  instructor  time  and  resource  utilization. 

(d)  Standardization.  CM!  contributes  to  the  standardization 
of  a  training  system  insofar  as  the  instructional  materials  must 
be  modularized  and  the  resources/equipment/facilities  are  part  of 
a  structured  scheduling  and  inventory  data  base  linked  to  the 
instructional  tasks/objectives.  Thus,  even  though  the  components 
of  a  CMI  system  (instructors,  facilities,  equipment)  may  undergo 
change  and  replacement,  the  system  itself  remains  the  same.  This 
type  of  standardization  of  a  training  situation  can  result  in 
cost  savings  in  terms  of  the  time  required  to  learn  or  modify  the 
training  system.  Furthermore,  this  type  of  standardization 
reduces  the  instructor-dependent  variability  of  a  training 
program,  particularly  in  the  testing  aspects. 

(e)  Flexibility.  In  addition  to  the  standardization  of  the 
training  situation  obtained  through  CMI,  a  degree  of  flexibility 
is  also  achieved.  Because  all  activities  are  in  terms  of 
instructional  tasks  and  skills  taught,  it  is  possible  to  replace 
resources/equipment  which  are  obsolete,  broken,  missing/lost, 
etc.,  without  a  major  reorganization  of  the  system.  This  could 
be  a  cost  savings  in  terms  of  reduced  system  development  or  main¬ 
tenance  time/expenses. 

(f)  Field  deployability.  Via  telecommunications,  it  is 
possible  to  conduct  CMI-based  training  at  sites  other  than  the 
main  training  center.  For  example,  the  Navy  has  conduced  CMI  at 
sea  via  satellite  transmission  from  land-based  training  sites. 

The  capability  to  train  personnel  in  the  field  rather  than  remove 
them  for  training  activities  could  mean  a  cost  savings  in  terms 
of  transportation  and  logistics  support. 

(g)  Expand ibil ity .  Because  the  system  is  standardized,  it 
is  possible  to  rapidly  increase  the  number  of  student  positions 
while  eliminating  (or  minimizing)  the  need  to  train  additional 
managers  or  add  additional  resources.  With  the  installation  of 
the  appropriate  hardware,  it  is  possible  to  completely  duplicate 
a  training  program  at  a  new  site  with  minimal  additional 
resources.  Thus  expanding  a  CMI  system  to  handle  more  students/ 
sites  tends  to  decrease  operating  costs  per  student  rather  than 
increase  them. 


4.2  Specific  Cost  Considerations  for  F-16  Environment 

Because  the  F-16  training  program  has  not  been  implemented 
yet  and  is  still  in  a  state  of  flux,  critical  data  needed  to 
generate  exact  cost  estimates  (e.g.,  number  of  students,  instruc¬ 
tors,  objectives,  simulators,  media,  training  sites)  is  either 
unavailable  or  unreliable.  Ideally  one  would  use  data  on  student 
throughput,  performance  levels,  resource  utilization  or 
instructor/student  ratios  with  existing  instructional  strategies 
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and  management,  and  compute  the  cost  savings  resulting  from 
individualized  instruction  and  computer  management.  Thus  in  the 
present  case  it  will  be  necessary  to  use  data  from  the  F-4  and 
F-15  training  programs  and  extrapolate  it  to  the  F-16  program. 

Table  3  provides  such  data  which  will  be  used  in  this 
section.  These  numbers  are  estimates  made  by  the  F-4  and  F-15 
Operations  Training  Development  (OTD)  teams  and  will  vary  some¬ 
what  with  circumstances  and  assumptions. 

The  predominant  cost  savings  which  result  from  CMI  systems 
in  military  and  industrial  settings  are  the  saved  student  and 
instructor  salaries  resulting  from  reduced  training  time  produced 
by  individualization.  For  example,  in  the  Navy  CMI  experience,  a 
course  time  reduction  of  30  percent  was  the  objective.  In  fact, 
an  actual  reduction  of  46  percent  in  course  duration  was 
achieved.  In  the  F-16  environment,  the  effects  of  time  savings 
are  not  likely  to  be  quite  as  dramatic  because  of  the  number  of 
simulation  sessions  and  flight  sorties  which  prevent  much 
compression  of  the  training  schedule.  Each  simulator  session  or 
flight  sortie  requires  briefing/debriefing,  associated  academics 
time,  and  aircraft  conf igur at ion/schedul ing .  However,  the  dura¬ 
tion  of  most  academic  instruction  can  be  reduced  by  individual¬ 
ized  scheduling.  Given  an  amount  of  academic  instruction  compar¬ 
able  to  F-4/F-15  and  a  similar  instructional  organization,  an 
average  10  percent  time  reduction  would  appear  reasonable.  This 
translates  into  savings  of  about  two  weeks  per  course  or  approxi¬ 
mately  25  student  full-time  equivalent  ( FTE)  salaries  for  the 
training  period. 

The  online  test  grading  and  student  records  capabilities  of 
the  CMI  system  will  amount  to  further  time  savings.  Every  test 
simulator  session  and  flight  sortie  requires  time  to  grade  and 
record  scores  and  provide  the  student  with  the  results.  The  F-4 
training  program  results  in  131  evaluation  situations  per  course 
(across  all  instructional  activities) ;  the  F-16  environment  is 
anticipated  to  have  about  200.  Each  of  these  evaluations  takes 
from  10  to  30  minutes  to  complete  properly  for  each  of  approxi¬ 
mately  250  students.  Assuming  an  average  of  20  minutes  per  eval¬ 
uation,  this  amounts  to  a  total  of  17,666  hours  of  instructor 
time  per  course.  (This  calculation  does  not  assume  that  instruc¬ 
tors  are  actually  doing  this,  but  that  this  is  the  amount  of  time 
which  should  b^  spent  on  this  evaluation/data  recording/prescrip¬ 
tive  activity.)  Consequently,  the  automatic  grading  capability 
alone  could  save  more  than  10  man-years  of  instructor  time  per 
course.  Furthermore,  this  is  a  conservative  estimate  because  the 
student  records  facility  means  that  routine  reports  and  special 
inquiries  can  be  produced  immediately  without  lengthy  and  time- 
consuming  efforts  on  the  part  of  instructors  or  support 
personnel . 

With  the  automated  support  of  the  CMI  recommended  in  this 
report,  the  number  of  scheduling  personnel  required  will  be 
reduced,  while  speed  and  efficiency  will  be  greatly  increased. 


22 


Table  3.  Data  on  F4/F15  Training  Programs 


Note  :  Estimates  of  real  numbers  by  F-4  and  F-15  ISD  teams  vary  widely  with  circumstances 
and  assumptions.  Figures  here  are  generally  accepted  estimates  by  those  teams. 


Manual  scheduling  methods  are  very  limited  in  terms  of  the  number 
of  elements  which  can  be  simultaneously  scheduled,  the  number  of 
scheduling  variables  which  can  be  accounted  for,  and  the  time  it 
takes  to  generate  original  and  revised  schedules.  It  is  a  rea¬ 
sonable  estimate  that  a  CMI  system  with  the  type  of  scheduling 
capabilities  recommended  could  reduce  the  administrative  person¬ 
nel  requirements  by  50  percent.  For  the  purpose  of  this  study,  a 
conservative  estimate  of  one-third  time  savings  will  be  used. 
Using  the  F-4  and  F-15  data  as  a  guideline,  it  is  estimated  that 
each  F-16  squadron  (three  squadrons  per  wing;  one  wing  per  base) 
will  require  two  scheduling  personnel  for  flight  and  simulation 
sorties  and  two  scheduling  personnel  for  academic  instruction. 
Scheduling  activities  are  half-time  activities  only.  Thus,  this 
amounts  to  a  total  of  six  scheduling  personnel  per  wing  per  base. 
Thus  a  conservative  savings  of  two  scheduling  assistants  per  base 
can  be  achieved,  even  though  the  scheduling  process  will  be  more 
efficient  and  operate  under  real-time  turnaround. 

It  should  be  pointed  out  that  the  F-16  training  program  has 
very  sophisticated  information  and  data  requirements.  This  is  a 
result  of  employing  a  mastery  model  of  instruction,  criterion- 
referenced  objectives  (CROs) ,  and  the  generation  of  competency 
profiles  for  student  performance.  This  means  that  diagnostic 
testing  and  student  "recycling"  (review  or  remediation)  will  be 
necessary.  It  will  be  absolutely  essential  that  the  data  record¬ 
ing  and  report  generation  capabilities  can  meet  these  sophisti¬ 
cated  demands.  Thus,  if  th.is  management  was  to  be  done  manually, 
the  F-16  program  would  need  considerably  more  administrative  sup¬ 
port  personnel  than  the  F-4/F-15  programs  (approximately  six  per 
squadron) .  A  conservative  estimate  would  be  a  doubling  of  needed 
personnel  (i.e.,  eight  to  10  per  squadron).  However,  CMI  systems 
typically  reduce  the  number  of  support  personnel  required;  the 
Navy  CMI  system  produced  a  70  percent  savings.  Thus,  CMI  support 
of  data  processing  will  amount  to  at  least  five  fewer  FTE  admin¬ 
istrative  personnel  per  squadron  or  15  fewer  per  base. 

To  summarize,  this  section  has  discussed  the  specific  cost 
savings  for  the  F-16  training  program  which  can  be  anticipated 
through  the  use  of  CMI.  This  includes  reduction  of  two  weeks  per 
course  duration,  or  25  student  salaries  for  the  training  period, 
10  man-years  of  instructor  time  per  course  due  to  automatic  test 
grading  and  student  records,  and  17  full-time  personnel  in  the 
area  of  scheduling  and  administrative  support.  All  of  these 
estimates  are  based  upon  conservative  assumptions.  Furthermore, 
there  are  additional  expected  benefits  of  CMI,  such  as  greater 
student  throughput  and  hence  greater  combat  readiness,  improved 
performance  levels  and  reduced  attrition,  and  improved  resource 
utilization  which  cannot  be  easily  captured  in  cost  figures  but 
which  will  ultimately  result  in  cost  savings  in  the  F-16  training 
program. 
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5.0  CONCLUSIONS  AND  RECOMMENDATIONS 


It  is  the  conclusion  of  this  study  that  the  F-16  training 
system  must  have  CMI  support  if  it  is  to  function  at  a  high  level 
of  efficiency.  This  includes  maximal  student  throughput,  maximal 
utilization  of  instructional  resources,  high  performance  level 
and  low  failure  rates,  and,  ultimately,  maximal  combat  effective¬ 
ness.  In  addition,  it  is  felt  that  a  CMI  system  is  essential  for 
satisfying  the  needs  of  the  total  training  management  system 
(TMS).  These  include  standardization,  flexibility,  self-monitor¬ 
ing,  high  reliability,  responsiveness,  problem  diagnosis/detec¬ 
tion,  and  large-scale  data  base  manipulation. 

The  preceding  sections  of  this  report  have  provided  the 
functional  specifications  of  the  CMI  system  needed  for  the  F-16 
training  system,  a  discussion  of  existing  systems  relevant  to  the 
F-16  environment,  and  a  cost/benefits  analysis.  This  section 
recommends  a  specific  CMI  system  in  terms  of  the  functional  spec¬ 
ifications,  existing  systems  survey,  and  cost/benef its  analysis. 

Table  4  summarizes  the  five  systems  surveyed  in  terms  of  the 
required  functional  capabilities,  their  operational  status, 
remote  capability,  training  status,  and  overall  cost.  As  far  as 
the  capabilities  are  concerned,  none  of  the  systems  surveyed 
provide  all  of  the  capabilities  needed,  although  VTS,  Navy  CMI, 
and  AIS  come  close.  Thus,  TICCIT  and  AFORMS  would  need  consider¬ 
able  enhancement  to  meet  all  of  the  F-16  CMI  needs.  With  regard 
to  implementation  status,  only  TICCIT  and  the  Navy  CMI  system  can 
be  considered  fully  operational.  This  means  that  further  devel¬ 
opment  time  is  required  before  AFORMS,  VTS,  or  AIS  could  be 
implemented  as  an  F-16  CMI  system.  While  all  five  systems  allow 
some  sort  of  teleprocessing  or  remote  access  capability,  TICCIT 
and  VTS  have  the  virtue  of  being  standard  microcomputer-based 
systems,  rather  than  large  time-shared  mainframes.  Large 
distributed  systems  have  the  advantage  of  allowing  remote  termin¬ 
als  and  shared  central  resources.  On  the  other  hand,  they  can 
also  lead  to  poor  system  reliability  and  degraded  performance 
resulting  from  system  saturation.  In  addition  to  improved  reli¬ 
ability,  standalone  systems  provide  greater  system  security. 

AFORMS,  TICCIT,  VTS,  and  AIS  have  all  been  applied  to  the 
domain  of  aircrew  training  while  the  Navy  CMI  system  has  not. 

This  is  an  important  consideration  insofar  as  overall  development 
time  will  be  reduced  for  systems  which  have  been  implemented  in 
an  aircrew  training  environment.  In  addition,  VTS  is  an  approved 
training  device. 

A  further  factor  has  to  be  considered  in  evaluating  these 
systems:  The  F-16  CMI  system  will  be  developed  in  the  context  of 
a  total  management  system  (TMS).  Thus  the  CMI  system  must  not 
only  provide  the  overall  capability  for  the  management  of 
individualized  student  programs  but  it  must  also  interface  with 
other  aspects  of  the  TMS,  e.g.,  an  author  management  system,  part 
task  trainers,  or  a  CAI  application. 
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Table  4.  Comparison  of  5  Systems  for  F-16  CMI  Suitability 


AFORMS 


I.  FUNCTIONAL  CAPABILITIES  TESTING 
Auto.  Test  Grading 
Diagnostic  Testing 
Test  Security 
Assess.  Instr.  Effect. 

Student  Feedback 

STUDENT  RECORDS 
Bibliographic  info. 

Report  Generation 
Sort  Capability 

SCHEDULING 
Track  Schedules 
Generate  Schedules 
Project  Schedules 

INVENTORY 
Personnel  Availability 
Facilities  Available 
Project  Resources 

PRESCRIPTIONS 
Individual  Prescriptions 
Resource  Prescriptions 


II.  OPERATIONAL  STATUS 
Completely  Operational 
Partially  Implemented 
Prototype 


III.  REMOTE  CAPABILITY 
Stand-alone 
Distributed 


IV.  TRAINING  STATUS 
Aircrew  Training 
Approved  Training  Device 


V.  COST 

High  (>  $1  Million) 
Low  (<  $1  Million) 


In  terms  of  cost  factors,  three  of  the  systems  (AFORMS, 

NAVY,  CMI ,  and  AIS)  require  rather  expensive  hardware  and  opera¬ 
tional  support,  while  the  comparable  costs  for  TICCIT  and  VTS 
are  relatively  low.  However,  in  the  case  of  AFORMS,  AIS  and 
possibly  TICCIT,  it  can  be  assumed  that  the  purchase  of  the  hard¬ 
ware  will  not  be  necessary,  i.e.,  that  it  would  already  be  avail¬ 
able  to  the  F-16  program.  In  the  case  of  AIS,  it  would  still  be 

necessary  to  account  for  the  high  cost  of  the  terminals  ($10K- 

$20K  per  cluster)  and  telecommunications  ($500  per  month  per 
cluster) . 

The  costs  associated  with  the  software  development  required 
varies  considerably  across  the  systems.  It  is  estimated  that 
AFORMS  will  require  a  rather  extensive  software  effort  in  terms 
of  the  specification,  implementation,  and  documentation  of  the 
extensions  and  modifications  needed  for  the  F-16  CMI  system,  as 
well  as  subsequent  work  to  extend  the  CMI  system  to  encompass  the 
full  TMS .  The  CMI  effort  alone  is  estimated  to  be  in  the  order 
of  three  man-years  taking  at  least  one  year  to  accomplish. 

TICCIT  development  work  would  be  relatively  small  given  the 
modularized,  stable,  and  well-documented  nature  of  the  system. 

The  work  required  to  add  the  necessary  scheduling  and  inventory 
components  is  estimated  to  be  in  the  order  of  six  man-months  of 
effort.  Navy  CMI  and  AIS  would  need  some  minor  software  conver¬ 
sion  work.  In  the  case  of  Navy  CMI,  it  would  be  a  matter  of 
enhancing  the  scheduling  and  inventory  capabilities;  in  the  case 
of  AIS,  it  would  be  necessary  to  "degrade"  the  system  by  reducing 
the  overhead  resulting  from  the  research  orientation  of  the 
system.  These  efforts  for  Navy  CMI  and  AIS  are  estimated  to  be 
in  the  order  of  three  to  four  man-months.  VTS  is  seen  as  only 
requiring  slight  implementation  changes  (e.g.,  changed  report 
headings) ,  all  of  which  would  be  included  in  the  installation 
package  provided  with  the  system. 

To  summarize  all  of  these  cost  considerations,  AFORMS, 
TICCIT,  and  VTS  are  within  an  acceptable  cost  range  based  upon 
the  assumption  that  the  hardware  for  AFORMS  and  TICCIT  is 
already  available.  If  this  assumption  is  not  met,  and  there  are 
some  grounds  for  believing  that  a  separate  system  may  be  needed 
for  F-16,  VTS  becomes  the  only  satisfactory  system  in  terms  of 
cost  factors. 

Weighing  all  of  these  factors  together,  it  is  the  recommen¬ 
dation  of  this  study  that  VTS  be  implemented  as  the  CMI  system 
needed  for  the  F-16  training  program.  VTS  is  able  to  meet  and 
exceed  all  of  the  requirements  specified,  is  relatively  inexpen¬ 
sive,  is  currently  in  use  in  aircrew  training,  and  has  the  advan¬ 
tages  of  both  distributed  and  stand-alone  capabilities.  It  also 
is  an  approved  military  training  device  and  has  interservice 
usage.  Most  importantly,  VTS  is  designed  to  support  a  training 
system,  rather  than  just  CMI  or  flight  administration  activities. 
This  will  allow  the  scope  needed  for  the  full-scale  F-16  environ¬ 
ment  . 
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Appendix  A 

PROGRAM  DESCRIPTIONS/SAMPLES  FOR  TICCIT 


MAKE  COURSE 

To  build  the  permanent  data  area  for  a  new  course,  containing 
information  about  its  structure.  Necessary  for  registration  and 
data  reduction  for  that  course.  Builds  a  file  called  PDAnnnsss 
where  nnn  is  a  3-digit  course  numbe;  and  sss  is  a  3-digit 
site  code. 

None . 

MAKE  COURSE 

This  initiates  the  following  dialog: 

(Entries  following  each  colon  (:)  are  user  responses . ) 

COURSE  NUMBER:  101 

(Valid  course  numbers  are  in  the  range  Q~  n-  655  . 

An  empty  input  terminates  the  program.) 

COURSE  MAP  MODULE  NUMBER  (DECIMAL):  4096 
(Valid  module  numbers  are  in  the  range  l£.  n—  4800G. 

Empty  entries  are  not  allowed.) 

HIGHEST  UNIT  NUMBER:  4 

(This  is  the  largest  number  used,  not  necessarily  the  number 
of  units  in  the  course.  Valid  unit  numbers  are  in  the  range 
04.  n  ^-30.  An  empty  entry  is  equivalent  to  0.  IMPORTANT: 

If  the  TICCIT  INTRO  UNIT  is  to  be  attached  to  the  course, 
the  highest  unit  number  must  be  increased  by  1  to  include 
it.) 

START  ARROWS  ON  SOX  NUMBER:  4 

(When  the  student  first  logs  on,  the  blue  arrows  on  the  MAP 
will  point  to  this  box.  An  entry  of  0  will  cause  the  arrows 
to  point  to  the  INTRO  box.) 

UNIT  1  STATUS  (R, Y,G  , U ,K ,C , W,  N):  K 

(A  permissible  entry  here  is  anyone  of  the  letters  shown  in 

parenthesis  above,  the  meaning  of  each  letter  is:  R  =  Red, 

Y  =  Yellow,  G  =  Qreen,  U  =  Blue,  K  =  Blacit,  C  -  Cyan, 

W  =  White ,  N  =  Nonexisting  unit.) 

HIGHEST  LESSON  NUMBER  IN  UNIT:  5 

(See  description  under  "HIGHEST  UNIT  NUMBER:".) 

UNIT  2  STATUS  (R,  Y,  G,  U,  K,  C,  W,  N):  N 
UNIT  3  STATUS  (R,  Y,  G,  U,  K,  C,  W,  N):  C 
HIGHEST  LESSON  NUMBER  IN  UNIT:  4 
UNIT  4  STATUS  (K ,Y ,G , U  , C  , W,  N):  X 
HIGHEST  LESSON  NUMBER  IN  UNIT:  3 

FOR  UNIT  1,  DO  ALL  6  LESSONS  EXIST  AND  IS  INITIAL  COLOR 
OF  ALL  EOXES  TO  BE  BLACK?  (Y  or  N):  Y 
(To  answer  Y  will  set  up  a  unit  with  6  lessons  existing  and 
each  cclored  black.) 


{ 


Format : 
Purpose : 


Switches : 
Asterisks : 
Example : 


1 


TERMINALUS  E 


TERMINALUSE  logfile  [ . . . logf ile^] 

To  build  in  the  disk  file  called  TERMINALUSE 
(no  extension)  a  chart  shewing  which  terminals 
were  used  during  the  day,  hew  long  they  were 
on,  and  at  what  tines  they  were  on.  Only 
those  terminals  which  were  actually  used  during 
the  day  are  included. 

/C  -  also  compress  the  data  vertically. 

Mot  permitted. 

TERMINALUSE  LOGO 30 2  .  <AM,PM> 

This  builds  a  terminal-use  chart  for  the 
2nd  cf  March  and  appends  it  to  the  disk 
file  TERMINALUSE.  The  chart  can  later  be 
printed  out  by  using  the  command  file  called 
TERMINALS  like  this: 

0  TERMINALS  @ 


A- 3 


BASIC  ITEM  ANALYSIS  REPORT 


All  practice  and  test  Items  are  gathered  into  the  item  data  base  by  using  the 
program  called  IGATHER.  The  gathered  information  can  be  printed  out  by  running 
ITEMS.  This  document  describes  the  report  thusly  printed  out. 

The  label  is  of  the  form  (segment  level  ^  (  item  v/here 

level  is  either  E,  M,  or  H  (for  practice  items)  or  1,  2,  or  3  (for  test  items). 


The  columns  labeled  #1  through  # 24  represent  the  occasion  codes  set  during 
answer  processing.  UN  is  #27  which  stands  for  unanticipated.  RT  represents 
a  "+"  in  the  occasion  code  field  on  the  answer  processing  form  (as  opposed  to 
,  and  TOT  is  the  total  number  of  times  the  item  was  tried. 

AV.LAT  is  the  average  amount  of  time  answering  an  item  (in  seconds).  %RT  is 
the  percentage  of  correct  answers  for  this  item. 


■.icr<!  V 

»»  W  L.  AL.  I 


Format:  WEEKLY 

WccKu (  1  n  o u z - ~  *  1 s 

Purpose:  To  print  the  weekly  student  report.  By  default,  the 

stuaents  will  be  sorted  by  name. 

S.vi tones:  /N  -  do  not  rebuild  AUSUM.OF 

Global:  /?  —  sort  by  r.ur.oer  of  lessons  passed. 

/'.!  -  sort  by  start  week. 

/7  -  sort  by  time-on-line. 

/D  -  sort  by  last  cay  on  system. 

/M  -  sort  by  tries  per  lessen  (mastery). 

/S  -  suppress  student  names  from  report. 

/O  -  suppress  student  numbers  from  recort 
Local:  None 

Asterisk:  Not  permitted 

Examples:  WEEKLY 

Er.ter  ccurse/sectlon  number:  00701 
Enter  course/ sec  tier,  r.umoer:  kJ 

R_ 

WEEKLY  REPORT 
R 

wr.ere  REPORT  contains  the  following: 

00701 

loan 

15SG3 

Comments:  The  program  will  ask  for  a  course-section  number  which. 

consists  of  5  digits,  the  first  2  being  tr.e  course  number, 
one  last  2  being  tr.e  section  number.  After  printing  tr.e 
report  of  zr.at  section,  it  will  as;<  again.  The  program 
is  terminated  by  entering  only  a  carriage  return. 

Only  one  sort  option  may  be  used  each  time  WEEKLY 
. is  run. 

The  file  AUSUM.SV  must  exist  to  run  WEEKLY. 
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Name: 


SORT 


Format:  SORT  soec-flle  error-file  input-file  ou tout-file 

Purpose:  To  sort  the  log  records  in  Input-file  into  the  order  specified 

by  the  fields  in  spec-file  and  place  the  results  in  output-file . 
If  an  error  is  detected  in  spec -file  ,  the  data  from  spec-file 
will  be  moved  to  error-file  and  an  arrow  will  be  inserted  pointlr. 
to  the  mistake.  Output  devices  $LPT  or  $TTO  may  be  used  for 
error-file . 

Switches;  None. 

Asterisk:  Not  permitted . 

Example:  SORT  CHAR  STTO  LOGO  OUTO 

where  CHAR  contains:  DT  ST  TM 

Sort  the  log  records  in  LOGO  first  according  to  the  date,  then 
according  to  student  number,  then  according  to  time,  and  then 
according  to  the  remaining  fields  in  default  sort  order.  The 
sorted  records  are  placed  in  OUTO. 

Notes:  The  input-file  must  be  either  random  or  contiguous,  or  SORT 

will  not  run  in  a  reasonable  amount  of  time .  Programs  TTOD 
and  SELECT  create  random  files;  therefore,  the  user  need  not 
not  normally  be  concerned  with  file  structure . 

An  intermediate  temporary  file  named  SORT.TF  must  exist  in 
the  default  directory.  It  may,  however,  be  a  link  to  a  file  in 
another  directory  (e.g.  DP4).  If  SORT.TF  is  a  link,  the 
directory  in  which  the  resolution  file  resides  will  be  auto¬ 
matically  initialized.  SORT.TF  (or  its  resolution)  must  be 
a  contiguous  file  and  should  be  created  like  this: 

CCONT  SORT.TF  n  where  n  is  the  number  of  disk 

sectors  to  be  allocated  to  SORT.TF.  SORT.TF  need  not  be 
created  more  than  once:  it  is  not  deleted  by  SORT,  and 
can  be  reused  any  number  of  times.  A  maximum  value  for 
n  (allocate  size  for  CCONT)  may  be  calculated  like  this; 
n  -  ( input-file  size)  /450 

SORT  executes  a  program  chain  to  another  program  SORT2.SV 
which  must  also  exist  in  the  default  directory.  A  small 
communication  file  SORT. CM  is  created  by  SORT  and  may  be 
deleted  subsequent  to  successful  execution  of  SORT. 
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SAMPLE  FORMS  AND  OUTPUT  FROM  NAVY  CMI  SYSTEM 
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Name: 


SORT  (Continued) 


Notes: 


(Continued) 

Following  are  the  fields  allowed  in  the  sort 
characteristic  file  together  with  their  mnemonics 
which  must  be  used.  They  are  given  in  default 
sort  order.  After  sorting  on  the  given  fields, 
SORT  will  then  sort  on  the  remaining  fields  in 
default  sort  order. 

Field  Mnemonic 


Date  DT 
Student  number  ST 
Time  TM 
Course  CR 
Unit  UN 
Lesson  LS 
Segment  SG 
Function  FN 
Part  PT 
Page  PG 
Item  IT 
Length  of  record  LG 
Latency  LT 
Port  PO 
Response  RE 
Flag  #1  FI 
Flag  #2  F2 


Flag  #24  F24 

Unanticipated  F27 

Spell  SP 

Code  CD 

Mode  MD 


Note:  Spaces  and  carriage  returns  may  be 
inserted  or  left  out  anywhere  in  the  string 
of  mnemonics  in  the  characteristic  file. 
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SAMPLE  "WEEKLY"  PRINTOUT 
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MAKECOURSE  (Continued) 


Examples: 


Errors: 


r.f’--'  ■ 


(Continued) 


FOR  UNIT  3,  DO  ALL  4  LESSONS  EXIST  AND  IS  INITIAL  COLOR 
OF  ALL  BOXES  TO  BE  BLACK?  (Y  or  N):  N 
(When  the  user  answers  "N",  the  program  will  inquire  about 
each  lesson  individually.) 

LESSON  1  STATUS  (R,  Y,  G,  U,  K,  C,  W,  N):  K 

(See  description  under  "UNIT  1  STATUS".) 

LESSON  2  STATUS  (R,  Y ,  G ,  U ,  K,  C ,  W ,  N):  W 

LESSON  3  STATUS  (R,  Y,  G,  U,  K,  C,  W,  N):  N 

LESSON  4  STATUS  (R,  Y,  G,  U,  K,  C,  W,  N):  K 

CREATED  PDA101 
COURSE  NUMBER:  999 

(This  begins  the  dialog  for  another  course.) 

HIGHEST  UNIT  NUMBER:  0 

(A  course  with  0  units  must  be  a  special  non-map  non-primary 
instruction  course  such  as  the  system  menu  used  for  data  entry.) 
CREATED  PDA999  000 
COURSE  NUMBER: 

R 

STATIC. SV  DOES  NOT  EXIST !  !  (This  file  is  used  by  the  program 
and  must  exist  to  run  it.) 

INVALID  COURSE  NUMBER 
COURSE  nnn  ALREADY  EXISTS 
INVALID  MODULE  NUMBER 
INVALID  LESSON  NUMBER 


P-17 


REGISTER 


* _ «  ...... 

Format: 

Purpose: 

Swi tches : 
Asterisk: 
Example: 


Comments : 


REGISTER  filename 

To  add  and  drop  students  from  the  authorization  directory, 
etc.  Also,  to  change  information  about  a  student  who  is 
already  registered  for  a  course. 

None 

Not  Permitted 
REGISTER  REG! 

Where  REG1  contains: 

1  537219614  JAMES,  WILLIAM  R 

2  927341129  CLAYTON,  WAYNE 

This  adds  William  James  to  course  001,  section  04 
and  it  drops  Wayne  Clayton  from  course  013. 

Valid  fields  in  the  registration  file  are  described  in 
the  documentation  of  the  on-line  registration  file  builder. 
This  program  also  updates  data  reduction  data  bases;  see 
notes  at  beginning  of  this  section.  However,  students 
registered  for  section  0  of  any  course  do  not  appear  in 
data  bases  or  reports. 


001  04 

013 
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OPERATIONS  SYSTEMS  OFFICE  OPERATIONS  SYSTEMS  OFFICE 


NOTE  REMOTE  LINE  PRINTERS  AND  OPTICAL 
SCAN  CAPABILITIES  WILL  BE  CONSIDERED  DURING 
FOLLOW-ON  ENHANCEMENT  OF  THE  SYSTEM  IF 
THEY  CAN  BE  ECONOMICALLY  JUSTIFIED 


Figure  2.  Complete  Versatile  Training  System  (VTS) 


Table  3.  Data  on  F4/F 15  Training  Programs 


Note  :  Estimates  of  real  numbers  by  F-4and  F-15  ISD  teams  vary  widely  with  circumstances 
and  assumptions.  Figures  here  are  generally  accepted  estimates  by  those  teams. 


Table  4.  Comparison  of  5  Systems  for  F-16  CMI  Suitability 


NAVY 

AFORMS  TICCIT  VTS  CMI 


I.  FUNCTIONAL  CAPABILITIES  TESTING 
Auto.  Test  Grading 
Diagnostic  Testing 
Test  Security 
Assess.  Instr.  Effect. 

Student  Feedback 

STUDENT  RECORDS 
Bibliographic  info. 

Report  Generation 
Sort  Capability 

SCHEDULING 
Track  Schedules 
Generate  Schedules 
Project  Schedules 

INVENTORY 
Personnel  Availability 
Facilities  Available 
Project  Resources 

PRESCRIPTIONS 
Individual  Prescriptions 
Resource  Prescriptions 


II.  OPERATIONAL  STATUS 
Completely  Operational 
Partially  Implemented 
Prototype 


III.  REMOTE  CAPABILITY 
Stand-alone 
Distributed 


IV.  TRAINING  STATUS 
Aircrew  Training 
Approved  Training  Device 


V.  COST 

High  (>  $1  Million) 
Low  (<  $1  Million) 


